An Explicit Convergence Rate for Nesterov's Method from SDP

نویسندگان

  • Sam Safavi
  • Bikash Joshi
  • Guilherme França
  • José Bento
چکیده

The framework of Integral Quadratic Constraints (IQC) introduced by Lessard et al. (2014) reduces the computation of upper bounds on the convergence rate of several optimization algorithms to semi-definite programming (SDP). In particular, this technique was applied to Nesterov’s accelerated method (NAM). For quadratic functions, this SDP was explicitly solved leading to a new bound on the convergence rate of NAM, and for arbitrary strongly convex functions it was shown numerically that IQC can improve bounds from Nesterov (2004). Unfortunately, an explicit analytic solution to the SDP was not provided. In this paper, we provide such an analytical solution, obtaining a new general and explicit upper bound on the convergence rate of NAM, which we further optimize over its parameters. To the best of our knowledge, this is the best, and explicit, upper bound on the convergence rate of NAM for strongly convex functions.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

An Explicit Rate Bound for the Over-Relaxed ADMM

The framework of Integral Quadratic Constraints of Lessard et al. (2014) reduces the computation of upper bounds on the convergence rate of several optimization algorithms to semi-definite programming (SDP). Followup work by Nishihara et al. (2015) applies this technique to the entire family of overrelaxed Alternating Direction Method of Multipliers (ADMM). Unfortunately, they only provide an e...

متن کامل

A geometric alternative to Nesterov's accelerated gradient descent

We propose a new method for unconstrained optimization of a smooth and strongly convex function, which attains the optimal rate of convergence of Nesterov’s accelerated gradient descent. The new algorithm has a simple geometric interpretation, loosely inspired by the ellipsoid method. We provide some numerical evidence that the new method can be superior to Nesterov’s accelerated gradient descent.

متن کامل

An inexact alternating direction method with SQP regularization for the structured variational inequalities

In this paper, we propose an inexact alternating direction method with square quadratic proximal  (SQP) regularization for  the structured variational inequalities. The predictor is obtained via solving SQP system  approximately  under significantly  relaxed accuracy criterion  and the new iterate is computed directly by an explicit formula derived from the original SQP method. Under appropriat...

متن کامل

Convergence analysis of the sinc collocation method for integro-differential equations system

In this paper, a numerical solution for a system of linear Fredholm integro-differential equations by means of the sinc method is considered. This approximation reduces the system of integro-differential equations to an explicit system of algebraic equations. The exponential convergence rate $O(e^{-k sqrt{N}})$ of the method is proved. The analytical results are illustrated with numerical examp...

متن کامل

A Survey of Numerical Methods for Nonlinear Semidefinite Programming

Nonlinear semidefinite programming (SDP) problems have received a lot of attentions because of large variety of applications. In this paper, we survey numerical methods for solving nonlinear SDP problems. Three kinds of typical numerical methods are described; augmented Lagrangian methods, sequential SDP methods and primal-dual interior point methods. We describe their typical algorithmic forms...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • CoRR

دوره abs/1801.04492  شماره 

صفحات  -

تاریخ انتشار 2018